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I. Basis of the report 



1 . With regard to the elements of the international application: 4 
| x | the international application as originally filed 
the description: 

1-3 



pages 
pages 
pages 



-32 



NONE 



, as originally filed 



, filed with the demand 



NONE 



, filed with the letter of 



| x| tne claims: 

pages 33-44 

pages NONE 

pages NONE 

pages 



, as originally filed 

as amended (together with any statement) under Article 19 
, filed with the demand 



NONE 



, filed with the letter of 



| x| tne drawings: 



pages 
pages , 
pages 



1-12 



, as originally filed 



NONE 



, filed with the demand 



NONE 



the sequence listing part of the description: 

pages NONE 

pages 

pages 



, filed with the letter of . 



NONE 



, as originally filed 

, filed with the demand 



NONE 



, filed with the letter of . 



2. With regard to the language, all the elements marked above were available or furnished to this Authority in the language in which 
the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language which is: 

1 | the language of a translation furnished for the purposes of international search (under Rule 23.1(b)). 
| | the language of publication of the international application (under Rule 48.3(b)). 

| | the language of the translation furnished for the purposes of international preliminary examination (under Rules 55.2 and/ 
or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the international 
preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in printed form. 

| | filed together with the international application in computer readable form. 

| | furnished subsequently to this Authority in written form. 

| | furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in the 
international application as filed has been furnished. 

I I The statement that the information recorded in computer readable form is identical to the writen sequence listing has 
1 — ' been furnished. 

4 | x| The amendments have resulted in the cancellation of: 

0 the description, pages NONE 

the claims, Nos. NQNE 

fx] the drawings, sheets/fig NONE 



5. Q^] This report has been drawn as if (some of) the amendments had not been made, since they have been considered to go 

beyond the disclosure as filed, as indicated in the Supplemental Box (Rule 70.2(c)).** 
* Replacement sheets which have been furnished to the receiving Office in response to an invitation under Article 14 are referred to 

in this report as "originally filed " and are not annexed to this report since they do not contain amendments (Rules 70. 1 6 

and 70.17). 

**Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this report. 
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V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 



1. statement 

Novelty (N) Claims ±6± YES 

Claims NONE NO 

Inventive Step (IS) Claims NONE YES 

Claims ±±* NO 

Industrial Applicability (IA) Claims ±^± YES 

Claims NONE NO 



2. citations and explanations (Rule 70.7) 

1. Claims 1-15, 20-34, 39-51, 56-61, 62/3, 62/4, 63/22, 63/23, 64/40, and 64/41 lack an inventive step under PCT 

Article 33(3) as being obvious over Dalziel et al (U.S. 5,579,444) in view of Nasar et al (U.S. 5,144,685), 

Dalziel et al disclose a system for generating a description record from image information, comprising: 

at least one image input interface for receiving said image information (Figure 2, Camera 12); a computer processor coupled 
to said at least one image input interface for receiving said image information therefrom, processing said image information 
by performing image object extraction processing to generate image object descriptions from said image information, processing 
said generated image object descriptions by object hierarchy construction and extraction processing to generate image object 
hierarchy descriptions (Figure 4; Column 12, Lines 1-25), and processing said generated image object descriptions by entity 
relation generation processing to generate entity relation descriptions, wherein at least one description record including said 
image object descriptions, said image object hierarchy descriptions and said entity relation descriptions is generated to represent 
content embedded within said image information (Figures 23 and 24; Column 39, Lines 64-67, Column 40, Lines 1-49. Object 
hierarchy description includes identifying oranges and apples, and color of the objects, i.e., red apples and green apples. Entity 
relation description identifies the relative position of the objects, i.e., object 1 is be;ow object 2.); and a data storage system, 
operatively coupled to said processor, for storing said at least one description record. Dalziel et al do not explicitly disclose 
generating entity relation graph descriptions for the identified objects. Nasar et al disclose a landmark recognition system 
compriisng generating entity relation graph descriptions (Figure 10; Column , Lines . The entity relation graph depicts 
positional relationship between the gate and the road in the landmark image of Figure 13.). 

It would have been obvious to a person of ordinary skil in the art at the time the invention was made to modify Dalziel et al 
(Continued on Supplemental Sheet.) 
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CLASSIFICATION: 

The International Patent Classification (IPC) and/or the National classification are as listed below: 
IPC(7): and US CI.: G06K 9/00, 9/34, 9/46, 9/56, 9/36, 9/60; G06F 3/00; G09C 5/00; H04N 7/12; G06F 7/00, 15/00* 
AND 

US CL.: 382/164, 165, 173, 181, 190, 195, 205, 232, 237. 304; 707/1, 3, 104, 514; 345/114, 328; 348/408; 706/55 



V. 2. REASONED STATEMENTS - CITATIONS AND EXPLANATIONS (Continued): 

invention according to the teachings of Nasar et al to generate entity relation graph descriptions for the identified objects 
because the symbolic characterization of the features using relative image information (image entity relation graph) will 
provide a better abstraction of the image and a framework for knowledge-based reasoning (Nasar et al; Column 8, Lines 



Regarding Claim 2, Dalziel et al further disclose the system of claim 1, wherein said image object extraction 
processing and said object hierarchy construction and extraction processing are performed in parallel (Figure 4, Task parallel 
Processing; Column 12, Lines 5-19). 

Regarding Claim 3, Dalziel et al further disclose the system of claim 1, wherein said image object extraction 
processing comprises: image segmentation processing to segment each image in said image information into regions within said 
image (Figure 2, Image Segmenter 14); and feature extraction and annotation processing to generate one or more feature 
descriptions for one or 

more said regions (Figure 2, Tracker, Sketcher, ranger 16a-16c, Reeognizer 18; Column 10, Lines 58-67, Column 11, Linesl- 
6); whereby said generated image object descriptions comprise said one or more feature descriptions for one or more said 
regions (Figure 16; Column 30, Lines 15-41). 

Regarding Claim 4, Dalziel et al further disclose the system of claim 3, wherein said one or more feature 
descriptions are selected from the group consisting of media features, visual features, and semantic features (Column 30, Lines 
15-41. Feature descriptor green is selected from visual features of the object.). 

Regarding Claim 5, Dalziel et al further disclose the system of claim 4, wherein said semantic features are further 
defined by at least one feature description selected fre-a the group consisting of who, what object, what action, where, when, 
why, code downloading, and text annotation (Figure 24; Column 40, Lines 5-8. The semantic feature for Object 1 is "what 
object".). 

Regarding Claim 6, Dalziel et al further disclose the system of claim 4, wherein said visual features are further 
defined by at least one feature description selected from the group consisting of color, texture, position, size, shape, motion, 
code downloading, and orientation (Figure 24; Column 40, Lines 38-48. The visual feature for Objects 2 and 3 is "position". 
Object 2 is apart from Object 1 at points c and d.). 

Regarding Claim 7, Dalziel et al further disclose the system of claim 4, wherein said said media features are further 
defined by at least one feature description selected from the group consisting of file format, file size, color representation, 
resolution, data file location, author, creation, scalable layer, code downloading, ani modality transcoding (Figure 16; Column 
30, Lines 29-38. The media feature for Region A is "green".). 

Regarding Claim 8, Dalziel et al further disclose the system of claim I, wherein said object hierarchy construction 
and extraction processing generates image object hierarchy descriptions of said image object descriptions based on visual 
feature relationships of image objects represented by said image object descriptions (Figure 24; Column 40, Lines 38-48. The 
visual feature for Objects 2 and 3 is "position". Object 2 is apart from Object 1 at points c and d.). 

Regarding Claim 9, Dalziel et al further disclose the system of claim 1, wherein said object hierarchy construction 
and extraction processing generates image object bier: chy descriptions of said image object descriptions based on semantic 
feature relationships of image objects represented by said image object descriptions (Figure 24; Column 40, Lines 5-8. The 
semantic feature for Object 1 is "what object".). 

Regarding Claim 10, Dalziel et al further di. close the system of claim 1, wherein said object hierarchy construction 
and extraction processing generates image object hierarchy descriptions of said image object descriptions based on media 
feature relationships of image objects represented by said image object descriptions (Figure 16; Column 30, Lines 29-38. The 



10-13), 
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media feature for Region A is "green".). 



Regarding Claim 11, arguments analogous to those presented for Claims 8-10 are applicable to Claim 11. 



Regarding Claim 12, Dalziel et al further disclose the system of claim 1, wherein said object hierarchy construction 
and extraction processing generates image object hierarchy descriptions of said image object descriptions based on relationships 
of image objects represented by said image object descriptions, wherein said image object hierarchy descriptions have a 
plurality of hierarchical levels (Figures 24 and 26; Column 40, Lines 28-49). 

Regarding Claim 13, Dalziel et al further disclose the system of cl?im 12, wherein said image object hierarchy 
descriptions having a plurality of hierarchical levels comprise clustering hierarchies (Figure 26; Column 42, Lines 28-43). 

Regarding Claim 14, arguments analogous to those presented for Claims 8-10 are applicable to Claim 14. 

Regarding Claim 15, Nasar et al further disclose the system of claim 1, wherein said entity relation graph generation 
processing generates entity relation graph descriptions of said image object descriptions based on relationships of image objects 
represented by said image object descriptions, wherein said relationships are selected from the group consisting of visual 
feature relationships, semantic feature relationships and media feature relationships (Figure 10; Column 8, Lines 50-68). 

With regards to Claims 20 and 39, arguments analogous to those presented for Claim 1 are applicable to Claims 20 

and 39. 

With regards to Claims 21 and 40, arguments analogous to those presented for Claim 2 are applicable to Claims 21 

and 40. 

With regards to Claim 22, arguments analogous to those presented for Claim 3 are applicable to Claim 22. 

With regards to Claims 23 and 41, arguments analogous to those presented for Claim 4 are applicable to Claims 23 

and 41. 

With regards to Claims 24 and 42, arguments analogous to those presented for Claim 5 are applicable to Claims 24 

and 42. 

With regards to Claims 25 and 43, arguments analogous to those presented for Claim 6 are applicable to Claims 25 

and 43. 

With regards to Claims 26 and 44, arguments analogous to those presented for Claim 7 are applicable to Claims 26 

and 44. 

With regards to Claims 27 and 45, arguments analogous to those presented for Claim 8 are applicable to Claims 27 

and 45. 

With regards to Claims 28 and 46, arguments analogous to those present* ' for Claim 9 are applicable to Claims 28 

and 46. 

With regards to Claims 29 and 47, arguments analogous to those presented for Claim 10 are applicable to Claims 29 

and 47. 



With regards to Claims 31 and 48, arguments analogous to those presented for Claim 12 are applicable to Claims 31 

and 48. 

With regards to Claims 32 and 49, arguments analogous to those presented for Claim 13 are applicable to Claims 32 

and 49. 

With regards to Claims 33 and 50, arguments analogous to those presented for Claim 14 are applicable to Claims 33 

and 50. 



With regards to Claim 30, arguments analoi )us to those presented for Claim 11 are applicable to Claim 30. 



Form PCT/IPEA/KW (Supplemental Box) (July 19.98)* 



INTERNATIONAL PRKlMlNARY EXAMINATION RETORT 



In^^^Bonal application No. 
PCT/US99A2G1127 



Supplemental Box 

(To bo used when the spare in any of the preceding boxes is not sufficient) 



Continuation of: Boxes I - VIII Sheet 12 

With regards to Claims 34 and 51, arguments analogous to those presented for Claim 15 are applicable to Claims 34 

and 51. 

Regarding Claim 56, Dalziel et al further disclose the system of Claim 12, wherein said image object hierarchy 
descriptions having a plurality of hierarchical levels are configured to comprise multiple levels of abstraction hierarchies 
(Figure 26; Column 42, Lines 28-43). 

With regards to Claims 57, 59 and 61, arguments analogous to those presented for Claim 11 are applicable to Claims 
57, 59 and 61. 



and 60. 



With regards to Claims 58 and 60, arguments analogous to those presented for Claim 56 are applicable to Claims 58 



Regarding Claims 62/3 and 62/4, Nasar et al further disclose a landmark recognition system wherein one or more 
feature descriptions include pointers to extraction and matching code in order to facilitate code downloading (Figure 2; Column 
6, Lines 3-14). 

With regards to Claims 63/22 and 63/23, arguments analogous to those presented for Claims 62/3 and 62/4 are 
applicable to Claims 63/22 and 63/23. 



With regards to Claims 64/40 and 64/41, arguments analogous to those presented for Claim 62/3 and 62/4 are 
applicable Claims 4/40 an 64/41. 

2. Claims 16-19, 35-38 and 52-55 lack an inventive step under PCT Article 3(3) as beinng obvious over Dalziel et al (U.S. 
5,579,444) further in view of Nasar et al (U.S. 5,144, 85) and Tuong Dao (An Indexing Model for Structur Documents to 
Support Queries on Content, Structure and Attributes, IEEE Paper ISBN: 0-8186-8464-X). 

Regarding Claim 16, neither Dalziel et al nor Nasar et al disclose the system of claim 1, further comprising an 
encoder for receiving and encoding said image object descriptions into encoded description information. Tuong Dao disclose 
an indexing model for structured documents comprising an encoder for receiving and encoding document descriptions into 
encoded description information (Page 92, Section 2.2.3), wherein said data storage system is operative to store said encoded 
description information as said at least one description record (Page 93, Section 3.1). It would have been obvious to a person 
of ordinary skil in the art at the time the invention was made to modify Dalziel et al and Nasar et al combination according to 
the teachings of Tuong Dao to include an encoder for receiving and encoding the image object descriptions into encoded 
description information because it will expedite retrieving the desired documents and will implement the process more 
securely. 

Regarding Claim 17, arguments analogous to those presented for Claims 11 and 16 are applicable to Claim 17. 

Regarding Claim 18, neither of Dalziel et al, Nasar et al and Tuong Dao disclose the system of claim 17, wherein 
said encoder comprises a binary encoder. Utilization of binary encoders are extremely well known in the art. It would have 
been obvious to a person of ordinary skil in the art at the time the invention was made to use a binary encoder for receiving 
anc encoding the image object descriptions into encoded description information because it is the simplest and most 
convetional encoder routinely implemented in the art. 

Regarding Claim 19, Tuong Dao further disclose the system of claim 17, wherein said encoder comprises an XML 
encoder (Page 88, Column 1, Lines 1-3). 

With regards to Claims 35 and 52, arguments analogous to those presented for Claim 16 are applicable to Claims 35 

and 52. 

With regards to Claims 36 and 53, arguments analogous to those presented for Claim 17 are applicable to Claims 36 

and 53. 

With regards to Claims 37 and 54, arguments analogous to those presented for Claim 18 are applicable to Claims 37 

and 54. 
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With regards to Claims 38 and 55, arguments analogous to those presented for Claim 19 are applicable to Claims 38 

and 55. 

With regards to Claims 58 and 60, arguments analogous to those presented for Claim 56 are applicable to Claims 58 

and 60. 

With regards to Claims 59 and 61, arguments analogous to those presented for Claim 57 are applicable to Claims 59 

and 61. 

Applicants' response filed May 7, 2001 have been fully considered but they are not persuasive. 

Dalziel et al clearly disclose generating image object descriptions and image object hierarchy description from the 
image information as depicted in Figures 4, 22-24 and disclosed in Column 12, Lines 1-25; Column 39, Lines 64-67 , Column 
40, Lines 1-49. Based on the processed image information, type of the objects in the image (i.e., apple, orange and bowl) and 
the hierarchical information concerning relative location of the objects in the image have been disclosed. 

As it was previously stated, Dalziel et al do not explicitly disclose generating entity relation graph descriptions for 
the identified objects. Nasar et al disclose a landmark recognition system compriisng generating entity relation graph 
descriptions (Figure 10; Column , Lines . The entity relation graph depicts positional relationship between the gate and the 
road in the landmark image of Figure 13.). 

Combined teachings of Dalziel et al and Nassar et al disclose claimed invention cited in independent Claims because 
the symbolic characterization of the features using relative image information (image entity relation graph) will provide a 
better abstraction of the image and a framework for knowledge-based reasoning (Nasar et al; Column 8, Lines 10-13). 



NEW CITATIONS 

NONE 
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